Digital video capabilities can be incorporated into a wide range of devices, including digital televisions, digital direct broadcast systems, wireless communication devices, personal digital assistants (PDAs), laptop computers, desktop computers, digital cameras, digital recording devices, cellular or satellite radio telephones, and the like. Digital video devices can provide significant improvements over conventional analog video systems in creating, modifying, transmitting, storing, recording and playing full motion video sequences.
A number of different video coding standards have been established for coding digital video sequences. The Moving Picture Experts Group (MPEG), for example, has developed a number of standards including MPEG-1, MPEG-2 and MPEG-4. Other standards include the International Telecommunication Union Telecommunications (ITU-T) H.263 standard, QuickTime™ technology developed by Apple Computer of Cupertino Calif., Video for Windows™ developed by Microsoft Corporation of Redmond, Wash., Indeo™ developed by Intel Corporation, RealVideo™ from RealNetworks, Inc. of Seattle, Wash., and Cinepak™ developed by SuperMac, Inc. Furthermore, new standards continue to emerge and evolve, including the ITU-T H.264 standard and a number of proprietary standards.
Many video coding standards allow for improved transmission rates of video sequences by coding data in a compressed fashion. Compression can reduce the overall amount of data that needs to be transmitted for effective transmission of video frames. Most video coding standards, for example, utilize graphics and video compression techniques designed to facilitate video and image transmission over a narrower bandwidth.
The MPEG standards and the ITU-T H.263 and ITU-T H.264 standards, for example, support video coding techniques that utilize similarities between successive video frames, referred to as temporal or inter-frame correlation, to provide inter-frame compression. The inter-frame compression techniques exploit data redundancy across frames by converting pixel-based representations of video frames to motion representations. In addition, some video coding techniques may utilize similarities within frames, referred to as spatial or intra-frame correlation, to further compress the video frames. The video frames are often divided into smaller video blocks, and the inter-frame or intra-frame correlation is applied at the video block level.
In order to achieve video frame compression, a digital video device typically includes an encoder for compressing digital video sequences, and a decoder for decompressing the digital video sequences. In many cases, the encoder and decoder form an integrated “codec” that operates on blocks of pixels within frames that define the video sequence. As used in this disclosure, the term “codec” refers to an encoder, a decoder or an integrated encoder/decoder.
In the MPEG-4 standard, the codec typically divides a video frame to be transmitted into video blocks referred to as “macroblocks.” The ITU-T H.264 standard supports 16 by 16 video blocks, 16 by 8 video blocks, 8 by 16 video blocks, 8 by 8 video blocks, 8 by 4 video blocks, 4 by 8 video blocks and 4 by 4 video blocks. Other standards may support differently sized video blocks. For each video block in the video frame, an codec searches similarly sized video blocks of one or more immediately preceding video frames (or subsequent frames) to identify the most similar video block, referred to as the “best prediction.” The process of comparing a current video block to video blocks of other frames is generally referred to as motion estimation. Once a “best prediction” is identified for a current video block during motion estimation, the codec can code the differences between the current video block and the best prediction.
This process of coding the differences between the current video block and the best prediction includes a process referred to as motion compensation. Motion compensation comprises a process of creating a difference block indicative of the differences between the current video block to be coded and the best prediction. In particular, motion compensation usually refers to the act of fetching the best prediction block using a motion vector, and then subtracting the best prediction from an input block to generate a difference block. The difference block typically includes substantially less data than the original video block represented by the difference block.
After motion compensation has created the difference block, a series of additional steps can also be performed to further code the difference block and further compress the data. These additional steps may depend on the coding standard being used. In MPEG-4 compliant codecs, for example, the additional steps may include an 8×8 discrete cosine transform, followed by scalar quantization, followed by a raster-to-zigzag reordering, followed by run-length encoding, followed by Huffman encoding. A coded difference block can be transmitted with a motion vector that indicates which video block from the previous frame (or subsequent frame) was used for the coding. A codec of a receive device receives the motion vector and the coded difference block, and decodes the received information to reconstruct the video sequences.
The use of discrete video blocks in inter-frame and/or intra-frame compression can cause artifacts in the video sequence between adjacent video blocks. In particular, when a video frame is divided into video blocks for video coding, the edge of one video block may appear discontinuous with the adjacent edge of another video block. When this occurs, the video frame may appear “blocky,” which is highly undesirable. Transforms and quantization of video blocks can compound this undesirable blockiness effect in coded video frames.
In order to remove such “blockiness,” filtering can be performed on the video blocks to “smooth” the transitions between adjacent video blocks. Deblocking filters generally refer to the filters that are used to smooth the transitions between adjacent video blocks to reduce or eliminate blockiness artifacts. The ITU-T H.264 standard, for example, requires a deblocking filter as part of the in-loop coding. In this case, when filtering is part of the in-loop video coding, the previously coded frames used in motion estimation and motion compensation are filtered versions of such frames. For other standards that do not mandate a deblocking filter as part of the coding loop, post deblock filtering may still improve the quality of the video coding.